For TY 22, appeals are scored only for bulk no change.
Bulk appeal settlement requires (1) scoring appeals based on likelihoods of reductions justified and (2) determining the threshold at which appeals will be bulk settled.
For example, imagine two properties in two neighborhoods. Property one is a very heterogeneous neighborhood where nearby properties vary greatly in value and quality. Property two is a very homogeneous neighborhood where all properties are very similar. In such a scenario, we would score the reduction likelihood much higher for property one than property two. How (or if) the appeals would be bulk settled would be determined by the score threshold at or above which an appeal would be bulk settled. Depending on the scores and the threshold(s), either property one, both property one and two, or neither property may meet the criteria.
Typically, each appeal is evaluated by an analyst selecting a handful of comparable properties which meet a series of ranges or pre-defined criteria. These comps often include nearby properties which are similar in age, size, and construction to a target property. These comps will then be used to construct an estimate of normalized value to be applied to the target property such as price per square foot.
The core principle of appeal scoring is to estimate the variance/level of uniformity in assessments. In the current version, a higher score should represent greater confidence in the current valuation and corresponding no change decision. Proposed factors to consider include property level and neighborhood level factors.
Let’s score a pin as an example from 2021
(05273010150000).
Below is the information collected on this property.
Scores are out of 100 as follows:
Note: a change on 1st or 2nd cut leads to a score of 0
Our property here receives a score of 66.6. 50 pts since 1st/2nd cut had no change. 5 pts since prd was met. 11.6 based on the share of comps greater than the target property (47.5%).
Scoring the level of uniformity in appeals creates an ordered list of appeals by likelihood of non-uniformity. These scores can then be used in workflows through categorization (high, medium, or low variance appeal) or to issue bulk determinations based on thresholds.
The core principle of threshold setting is that some areas of Cook County are uniform enough that reductions cannot be justified except in extreme circumstances. While there will always be some level of variability in assessments, it would not make sense to grant a reduction of 1000 against a property with an assessment of 100,000 or any reduction below a certain percentage of a property’s value (say 5%). Properties which fall below this threshold would then only be justified in receiving a reduction if some extenuating circumstances exist such as very low building condition or some amount of uninhabitability.
Another aspect of thresholds is that they can be set at different levels by classes, townships/tris, or cuts. It may make sense to set the most generous threshold at 1st cut for a heterogeneous township and the least generous threshold for a very homogeneous township at 3rd cut.
Run on regression class residential property (class 202-212, 234, and 278). There were approximately 1.04 million parcels (55% of total) which received assessments in this category and 196,000 appeals filed (66% of unique 10 digit pins total appeals).
Below is a table which presents the results of the simulation for the appeals. Note that at this time, the simulation was done assuming that all appeals are 1st cut. This is the most difficult scenario and would not be recommend as the first course of action.
In order to determine a threshold, we want to look at the separability of the two classes as defined by the actual BOR decision from 2021 (change or no change). Below are a series of plots which show the distribution across various metrics.
As discussed, the current version of bulk appeals is that a threshold would be a decision boundary at which appeals could be automatically adjudicated no change. The current methodology is a rudimentary classification algorithm which can be evaluated utilizing the standard techniques.
A confusion matrix shows the classified result versus the truth. For our situation:
Let’s demonstrate this with an example threshold of 25.
We can see that we have a lot of mistaken predictions at this threshold. This is in part because our heuristic cannot adequately distinguish between the two classes with the limited information it has. Let’s analyze the grid above again.
(Prediction - Truth)
Situation 1: No Change - No Change 74249 40%
This is the ideal situation where an appeal would be ‘correctly’ classified as no change just as an analyst would.
Situation 2: No Change - Decrease 18852 10%
This is the worst case scenario where an appeal would be ‘incorrectly’ classified as no change even though an analyst would provide a decrease. Our goal by threshold setting is to avoid this.
Situation 3: Predicted Decrease 91982 50%
This is a situation where an analyst would evaluate the appeal. In most cases of this category, the analyst found a decrease warranted. A secondary goal is to minimize the number of false negatives where an appeal was predicted decrease but an analyst found no change.
For example, let’s set the threshold at 35 instead.
We can see that the number of ‘bad’ false positives has decreased, but the number of appeals to be manually evaluated (false negatives plus true negatives) increases.
Scores and the threshold value will be easily available to analysts. An accessible feedback form will allow analysts to flag mistaken scores or threshold values.
Generating change valuation determinations at this time would be non-uniform since it is mid-cycle. This methodology will be established throughout the coming months.
Other criteria, such as sales in the past three years, can also be implemented if specified.